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A round-up for Audit Committees 


We have been delighted that the first edition of our Round-up for Audit Committees has been 
well-received. Over the six months since then we have published a number of reports that may 
be of interest to Audit Committees and we hope you find this second edition equally useful. 


Looking across government, it seems to me that one of the biggest issues is the scale of reform 
and demands on the public sector, from the competition for scarce skills and capacity to the 
stresses on those delivering public services. In my recent blog post, A flexible, engaged approach 
to efficient public services, | set out some of these issues, which | discussed with The Strand 
Group in early February. 


In the last edition of our ‘Round-Up’ | encouraged you to respond to our survey and let us know your 
views on the NAO’s work. | thank all of you and your colleagues who have provided feedback — we 
have received the final report and are considering how we can respond to increase the value we can 
deliver through our work. At any time, we invite you to contact us with ideas about further ways we 
can do more to help you and help improve public services. 


Sir Amyas C E Morse 
Comptroller & Auditor General 
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Governance, oversight and risk management [! PEE actors 
Many of our recent reports address key aspects of the interrelated elements of diay about a Monia, etait 
; ; governance report and revise 
governance, oversight and risk management. This section looks at some of the 
lessons and good practice that is identified in these reports for planning and 2 Realistic plans 5 Manage inter- 
developing governance arrangements, risk management, monitoring, reporting, and forecasts dependencies 
project management and oversight. 
3 Risk identification 6 Maximise value from 
and planning public services 


Success Factors (SFs) Good practice, lessons learnt and risks identified 


1 Clarity about Departments’ oversight of arm’s-length bodies! (ALBs) highlights the risks of tension between 
P * governance - roles, organisations and potential impact on services when departments and their ALBs are unclear about 
AD responsibilities their respective responsibilities - especially as ALBs operate with varying degrees of independence. 
and who is held to (See more on this report under SFs 3 and 6) 


account for what MEIE i 
Delivering the defence estate found there was convoluted governance of the contracting arrangements, 


leading to confused accountability, and resulting in poor performance going unchallenged. (See more 
on this report under SFs 2 and 4) 


The Department for Education’s management of a potential conflict of interest defines conflicts of 
interest and when they are likely to arise, such as during procurement. It outlines the sources of 
guidance on how they should be managed in government. Organisations are expected to apply these 
standards to develop and set up their own approaches, ensuring everyone is aware of and signs up to 
that approach. Individuals’ conflicts of interest should be declared and there should be clear rules on 
withdrawal from decision-making. Organisations should have checks and balances in place to reinforce 
compliance. Further details are in our earlier report, Conflicts of interest, which also outlines the risks 
and includes public sector examples. 


1 On 24 February 2017 the government published Partnerships between departments and arm’s-length bodies: Code of Practice 
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Success Factors (SFs) 


Good practice, lessons learnt and risks identified 


2 Realistic plans 
and forecasts, 

with reasonable 
assumptions 

and clear linkage 
between actions and 
outcomes desired 


Introduction of the new state pension concludes that implementation so far represents value for 
money, attributing this to the Department for Work and Pensions’ mitigation of project risks by 
developing a contingency plan, undertaking scenario analysis and involving operational teams in 
reviewing planning assumptions. 


Delivering value through the apprenticeships programme highlights the risk of not achieving a 
programme’s objectives without a clear strategy for doing so — in this case, it’s not clear how having 
more apprenticeships will increase UK productivity, nor how an appropriate mix of apprentices will be 
achieved in the programme. (See more on this report under SFs 3 and 5) 


Controlling the consumer-funded costs of energy policies — The Levy Control Framework found a failure 
to fully consider the uncertainty around the central forecasts or to define the associated risk appetite. 

If government had asked more explicitly “what if the forecasts or key assumptions are wrong?”, this 
might have prompted more robust design and monitoring of the Framework, thereby reducing the 
likelinood of significantly exceeding the budgetary cap. (See more on this report under SF 4) 


Managing the HMRC Estate found that a number of assumptions and plans were unrealistic, including 

a risky peak of programme activity in one year, creating real risks of disruption to the taxation system. 
There is also a need for HMRC to build flexibility into its 25 year property deals, especially as it has gone 
through three major changes in the last 12 years that have impacted on property arrangements. 


Modernising the Great Western railway found that Network Rail’s schedule and cost estimate for 
delivering infrastructure works was unrealistic. This contributed to costs of one part of the programme 
(electrification between Maidenhead and Cardiff) increasing from £1.1 billion to £2.8 billion between 
2013 and 2016. Network Rail made unrealistic assumptions about the productivity it would be able to 
obtain from introducing new technology. (See more on this report under SF 5) 
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Success Factors (SFs) 


Good practice, lessons learnt and risks identified 


2 Realistic plans 

and forecasts, 

with reasonable 
assumptions and 
clear linkage between 
actions and outcomes 
desired continued 


Review of the Thames Tideway Tunnel examined the evidence base that Department for Environment, 
Food and Rural Affairs (Defra) and the Environment Agency established as both the required standards 
to be met by any solution to address the problem of sewage overflows into the Thames Tideway, and 
as the basis for appraising options. Our report emphasises the importance of a robust evidence base 
when policymakers need to design interventions capable of meeting intended objectives, and when 
they need to forecast the likelihood of success and the estimated costs of different intervention options. 
In these situations it is important that organisations have a clear and up-to-date view of the impact that 
forecasting and modelling uncertainty could have on intervention designs and their associated costs. 
Equally, they need evidence to support a decision that the costs of reducing uncertainty further would 
not justify the benefits. (Gee more on this report under SF 3) 


Investigation into HMRC’s contract with Concentrix found that HMRC’s plan to save £1 billion over the 
three-year life of the contract was unrealistic. The contractor’s performance fell well below expectations 
in terms of both the original plan and a less ambitious, improvement plan. As summarised in our blog- 
post Setting up successful contracts the problems may have reflected the use of an outcome-based, 
payment-by-results model that had not been tested in the pilot. 


Delivering the defence estate is one of many reports identifying a risk that planned financial benefits 
won't be realised, which could put MOD’s ability to meet its strategic commitments at risk. (Gee more 
on this report under SF 4) 
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Success Factors (SFs) Good practice, lessons learnt and risks identified 


3 Identify risks Departments’ oversight of arm’s-length bodies? (ALBs) found that although the four departments 
and develop reviewed each assessed individual risks in ALBs, they did not routinely take account of system risk, 
contingency plans despite the fact that risks in one organisation can have consequences elsewhere in the family of ALBs, 


or in other departments. We also found that ALBs have an appetite for increased risk management 
support from departments. (See more on this report under SF 6) 


Delivering value through the apprenticeships programme found some indications that risk management 
was being approached as a paper exercise, rather than an effective mechanism for controlling the 
project. Moreover, those running the programme were not (at the time of our fieldwork) addressing the 
key risk that businesses and individuals may not behave as predicted, which can often happen during 
fast-paced changes to markets. We summarise the sort of behavioural risks that could relate to the 
reformed apprenticeship system. (See more on this report under SF 5) 


The Equipment Plan 2016-2026 found that the risks to the affordability of the Ministry of Defence (MOD) 
Equipment Plan are greater than at any point since reporting began in 2012. Financial risks to the Plan 
relate to potential cost growth, including because of the proportion of large projects at an early stage 

of development and the MOD’s costing policy, which has historically underestimated the cost of such 
projects, combined with the risk of exchange rate movements. One of our recommendations is that the 
MOD should identify the current projects most at risk from cost growth, and should assess how these 
projects could be costed more realistically in the Plan. We also set out how project teams account for risk, 
covering ‘Risk inside costing’ and ‘Risk outside costing’, the latter including modelled and un-modelled 
risks. The Appendix sets out the testable assertions and key findings relating to the Plan regarding 
whether the cost of the Plan over the 10-year period is equal to or less than the available funding. 


Review of the Thames Tideway Tunnel has a chapter exploring the risks of this major project, 

and Defra’s arrangements aimed at mitigating these risks. Risk management is particularly important 
in this project, both for the taxpayer, as private sector delivery has been made financially viable by 
transferring some of the risks to the taxpayers; and for Thames Water’s customers, who are footing 
the bill for the project. 


2 On 24 February 2017 the government published Partnerships between departments and arm’s-length bodies: Code of Practice 
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Success Factors (SFs) 


Good practice, lessons learnt and risks identified 


3 Identify risks 
and develop 
contingency plans 
continued 


Investigation into the Department for International Develooment’s approach to tackling fraud highlights 
processes for managing the risk of fraud. Our report looks at DFID, the Foreign & Commonwealth 
Office (FCO) and the British Council in terms of the fraud cycle conceptual framework, namely: 
preventing fraud; detecting fraud; investigating fraud; fraud recovery and sanctions, and fraud reporting 
and lesson learning. The report sets out the three organisations’ processes and effectiveness, including 
DFID’s ‘three lines of defence’ model, involving front-line staff; a control and assurance team, and 
internal audit. For example, DFID has built the consideration of fraud risk into the process of setting up 
programmes; including conducting due diligence over delivery partners. (See this web-page for other 
reports on managing fraud and error.) 


4 Monitor, measuring 
performance, 
reporting and 
decision-making / 
revision of plans 


Controlling the consumer-funded costs of energy policies — The Levy Control Framework found that 
poor governance contributed to a delay in identifying potential cost over-runs and need for new 
controls. The Board responsible for joint department and Treasury oversight of the Framework had 
stopped meeting and arrangements for sharing information between officials were not good enough. 


Children in need of help or protection highlights the link, demonstrated by Ofsted reports, between poor 
recording, use and sharing of information, and consequent poor performance. (See more on this report 
under SF 6) 


Delivering the defence estate stresses that the more complex the risks and the longer the programme, 
the more important the need to report on both progress to date and projected performance against 
the long-term strategy and medium-term plans. (This report is also summarised in our blog-post 

Our defence estate — right size, right condition, right price?) 
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Success Factors (SFs) 


Good practice, lessons learnt and risks identified 


5 Take inter- 
dependencies within 
programmes and 
between organisations 
and initiatives into 
account in planning, 
risk management, 
monitoring and project 
management 


Delivering value through the apprenticeships programme found, during our fieldwork, that it was often 
unclear where teams had considered dependencies between different initiatives. By July 2016, a 
Programme Management Office was in place, looking at inter-dependencies and identifying a ‘critical 
path’ (the minimum feasible schedule for the work, including dependencies between key stages), as well 
as risks, governance and contingency plans for high risk elements. Reflecting the risk of unpredicted 
behaviour changes, the Department for Education and the Skills Funding Agency established a counter- 
fraud and gaming scrutiny group, although at the time of reporting they had not yet established the 
information needed to monitor key behavioural risks and spot signals that these risks may be maturing. 


Modernising the Great Western railway is a key example of a highly complex set of programmes that 
were poorly planned and delivered. The absence of a business case covering the whole modernisation 
programme prior to 2015 (two and a half years after the contract for new trains was signed) meant 
that inter-dependencies were not identified and managed. The absence of a critical path led to the 
infrastructure elements costing more than they needed to, and a further £330 million of costs to the 
Department for Transport (DfT) caused by poor management of the inter-dependencies between 
infrastructure, new trains and the franchise operating services. 


Since 2015, DfT and Network Rail have worked to address the failings, including establishing a 
programme board that allows key stakeholders in all three elements of the programme to be involved in 
decision-making. They have strengthened cost estimation, monitoring arrangements, governance, and 
collaboration with contractors and the wider rail construction industry. However, our report concludes that 
programme management information remains weak, undermining the programme board’s effectiveness. 
In particular, information about costs and schedules has not been based on an earned value management 
approach, in line with best practice for managing major programmes (although we understand 
development of this is now underway). 
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Success Factors (SFs) 


Good practice, lessons learnt and risks identified 


O 9 6 Ensure the oversight 
drives maximum value 
EL from public services 
+” and minimises 
inconsistencies 


Departments’ oversight of arm’s-length bodies? (ALBs) illustrates how opportunities to maximise value 

to the public can be missed if departments focus their oversight on ALBs’ financial and administrative 
issues, rather than the strategic and quality factors key to their achievement of their overarching objectives. 
More positively, all four departments reviewed were encouraging collaboration across their ALBs, such 

as bringing together senior ALB staff to consider current issues and collaboration on specific issues in 

a targeted way. ALBs felt collaboration was most effective where departments had encouraged sharing 
practice on a specific, targeted issue. 


Children in need of help or protection highlights inconsistencies between service providers, such as 
different threshold levels for accessing help and protection for vulnerable children. To improve services 
across the system as a whole, providers need to know what works. The Department for Communities 
and Local Government accepts that it needs to improve the system for sharing good practice, and has 
introduced an Innovation Programme and Partners in Practice initiative to develop new models of social 
work, and plans to create a ‘What Works Centre’ for social work. There are also other ways of sharing 
good practice including benchmarking tools; local authority level practice sharing and peer review; and 
Ofsted’s inspection reports and ‘getting to good’ seminars. 


Several of these issues will also be covered in two forthcoming reports: 


Delivering Carrier Strike will examine whether the Ministry of Progress with delivery of the road investment strategy (RIS) is the 
Defence has effective arrangements in place for delivering first in a series of studies looking at the Department for Transport 
‘Carrier Strike’ (which includes the Queen Elizabeth Class and Highways England’s progress in delivering value from their 
Carriers, the F35 Lightning II Aircraft and the Crowsnest radar investment in the strategic road network. It will focus on how well 
system), including effective governance, risk management and they planned and are managing the whole portfolio of projects. 


appropriate commercial arrangements. 


3 On 24 February 2017 the government published Partnerships between departments and arm’s-length bodies: Code of Practice 
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Useful 
resources 


| Further 
information 


This section summarises our reports, good practice guides, blog-posts and other resources that Audit Committees might find 
helpful. It includes a focus on service delivery and user experience, local services, and commercial and contracting issues. 


Award-winning good practice 


Our blog post Award winners’ 
secrets highlights what we liked 
about the projects that won 
awards regarding ‘Analysis and 
Use of Evidence’, ‘Customer 
Service’, ‘Supporting Growth’, 

and ‘Skills’ (a digital transformation 
example). It also highlights useful 
NAO resources. 


See also Building Public Trust 
Awards - Examples of good 
practice in annual reports 2015. 


Measuring performance 


Performance measurement by 
regulators is a good practice 
guide that aids improvement in 
performance measurement and 
reporting by everyone, especially 
regulators and other organisations 
seeking to deliver outcomes 
through third parties. 


It’s complemented by Performance 


measurement good practice criteria 


and maturity model (docx — 49KB) 
and summarised in our blog-post 
Measuring performance delivered 


Pensions 


There have been major reforms 
to pensions since 2010, including 
the Introduction of the new state 
pension and Automatic enrolment 
to workplace pensions. 


We have created an interactive 
Pensions Landscape site to explain 
the system and the roles of public 
bodies involved, reforms since 2010 
and challenges and risks across the 
pension landscape, Summarised in 
our blog-post Navigating a changing 


pensions landscape. 


through others. 


All our related reports are on our 
new page: Our work on pensions. 


EU-UK finances 


Briefing on EU-UK finances 
provides information about the 
financial relationship between 
the UK and the European Union 
(EU), intended to help Parliament 
scrutinise the government’s 
strategy for leaving the EU. 
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Devolution 


Our forthcoming report Planning for 
100% local retention of business 
rates will be a value-for-money study 
on the Department for Communities 
and Local Government’s plans to 
design and put in place 100% local 
business rates retention by the end 
of the current Parliament. 


Devolution and financial and 
service sustainability are focuses 
of Sir Amyas Morse’s talk to The 
Strand Group, as Summarised in 
his blog-post: A flexible, engaged 


approach to efficient public services. 


See also our earlier report, English 
devolution deals, and blog-post, 
Steering English devolution. 


Financial sustainability 


Four new reports can be found 
on our Financial and service 
sustainability web-page; on the 
NHS, the ambulance service,* 
schools financial sustainability® 
and schools capital funding.® 


These reports set out the current 
performance and the issues the 
responsible department needs 
to focus on for financial and 
service sustainability, including 
the challenges, risks and 
support needed. 


4,5,6 These reports include podcast interviews with the Director responsible for this report. 


Useful 
resources 


Public service markets 


Our blog-post Making public 
sector markets work explains the 
spectrum of public sector markets, 
i.e. where the state retains ultimate 
responsibility, but private, public and 
voluntary providers compete for the 
right to deliver public services and 
users often have a say in or choose 
the services they receive. It outlines 
what makes these markets 
effective, highlighting a range of 
NAO resources, including our 
recent Market Analytic Toolkit: for 
assessing public service markets. 


We have also blogged on 
Driving forward competition in 
the UK economy. 


Further 
information 


Protecting consumers 


Protecting consumers from scams, unfair 
trading and unsafe goods reviews the 
effectiveness of the consumer protection 
system, involving about 200 organisations. 
It sets out the roles, costs and activities 

of the key organisations, the system’s 
ability to identify and address consumer 
detriment, and outcome evaluation. 


Citizens Advice Bureau consumer 
service cases can also be explored in our 
interactive data visualisation. 


The report and issues are Summarised in 
our blog-post: Do you feel protected as 
a consumer? 


We will shortly report on: Vulnerable 
consumers in regulated industries, which 
include energy, water, communications 
and financial services. 
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Children 


Three blog-posts relate to our 
reports on children: 


Housing 


Local government reform Health and social care 


Overview: Local government sets out Health and social care integration 
facts about how local government is reviews how the Department 
constituted and funded, and findings of Health, the Department for 

from our work on local government, Communities and Local Government 
and summarises what the next 12 and NHS England are progressing 
months may look like. with their plans to help reduce 
pressures on the systems by 
integrating health and social care 
services at the local level, and the 
extent to which integration has 


Housing in England: overview looks 
at the ability of government to 
achieve its ambitions for increasing 
the housing supply, providing 
evidence to assess future policy 
making and implementation. 


Putting children first: Making theory 
work in practice applies the principles 
set out in our good practice guide, 
Managing business operations, to 
child protection services.’ 


The evidence behind the report can 
be explored at the local level in this 
interactive data visualisation. 


Local public service reform 
summarises our research into how 
these reforms are being pursued in 


Care leavers: engaging for solutions 


eight places in England, the enablers 


, benefitted patients. 
and barriers, and government 


sets out facts about care leavers 
and their lives, and focuses on the 
great value of involving customers in 


See also ‘Devolution’ and other 
local issues on the previous page. 


support. It identifies sources of help. Local support for people with a 
learning disability looks at the health 
and social care support available to Good services: child’s play? looks 
all those with learning disabilities, at the state of services for children 
and whether this support is in care and what central and local 
improving outcomes. governments need to do. 


finding solutions. 


Our blog-post Local service 
reform: is it all about the money? 
summarises this report and looks 
at whether less money means 
poorer services. 


7 The Children in need of help or protection report includes a podcast interview with the Director 
responsible for this report and an easy-read version of the report on which this blog-post is based. 


Our resources 


Ty» Overview ws Governance, oversight Useful 


Commercial & contracting 


Commercial and contract 7-phase contract life cycle: 
management: insights and 

emerging best practice is Key judgements for commercial 
an interactive publication relationships 


identifying insights from our 
assessments in well over 
100 reports. It develops 20 


insights into what works Commercial strategy 
well and less well across; ———=75775775777777777777- S) 
government through using (1) Make time to develop strategy = 


Commercial 
strategy 


case studies and identifying (2) Properly develop strategy 
emerging best practice. 


Commercial capability 


6 Tailor capability to risks and opportunities 


Commercial Q Clarify commercial and operational balance 
capability 


Q Maintain ‘organisational capability’ 


Market management & sourcing 


QO Be ‘an attractive client’ 


Q Be ‘an intelligent client’ Market 
management 
O Properly evaluate bids & sourcing 


O Keep up competitive tension 


[=| Further 


and risk management resources => Information 


Contract approach 


(10) Ensure shared understanding 


Contract ® Understand risks 
approach 


(12) Design performance measures that work 


Contract management 


(13) Manage your own obligations mi 
: l BR 
D Know what suppliers are doing 
Contract 
(15) Show what you care about management 


© Understand suppliers’ motivation 


Contract lifecycle 


(17) Plan for uncertainties 


Contract 
lifecycle 


© Work towards business as usual requirements 


Transition & termination 


(19) Think about contract end up front 


den 


Transition 
& termination 


ð Allow time to consider potential 
end of contract 


Our 
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Commercial & contracting 


judgements 
for commercial 
relationships 


Emerging best practice 


Commercial and contract 
management: insights and emerging 


i 


best practice (Summarised on the 
previous page) is accompanied by: 


e Our Good practice contract 
management framework, 
focusing on the activities that 
organisations should consider 
when planning and delivering 
contract management. 


e NAO’s contractual relationships 


resources 
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f and risk management 


The emerging best practice 
document is complemented by 
two blog-posts: 


Government’s contracts — 

new insights into best practice 
summarises the insights and 
guide, outlining ‘manage your 
own obligations’ as one example 
and highlighting issues such as 
open book accounting. 


Setting up successful contracts looks 


audit framework, which sets out 
the sort of questions that NAO’s 
auditors may ask at each of the 
seven stages of the audit cycle. 


at further insights identified from 
three more recent reports, including 
effective performance measures, 
ensuring the IT and processes work 
for contractors, and the challenges of 
incentivising good customer service, 
summarising our Investigation into 
HMRC’s contract with Concentrix. 


Useful 
resources 


Crown Commercial Service 


The Crown Commercial Service 
(CCS) was launched in April 2014 
to save government money by 
centralising the buying of common 
goods and services, but it quickly 
ran into difficulties. 


Our report sets out why and how 
CCS was created, its performance, 
the difficulties it’s encountered 

and progress in addressing these 
since 2015. We conclude that 

the strategic argument for joint 
buying across government remains 
strong and make a number of 
recommendations for CCS and 
Cabinet Office, including the need 
to clarify CCS’s mandate and role. 


Further 
information 


Running competitions 


Carbon Capture and Storage 
(CCS): the second competition 

for government support examines 
how the Department for Business, 
Energy & Industrial Strategy 
designed and ran the second 
competition for government 
support of CCS projects before its 
cancellation in January 2016. Our 
report includes an overview of the 
CCS technology and its challenges. 


Related resources can be found 
on our environmental sustainability 
web-page. 
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Further information 


Find useful resources on NAO’s website and blog: 


NAO blog: our experts summarise issues and challenges facing government, drawing on insight 
from across our reports to outline what you should look out for and how organisations have 
addressed issues. 


For example: Regulation through the looking glass: explores how well the government is 
progressing with its aim to cut £10 billion from businesses’ regulatory costs. 


The ‘Topics’ tab on our website links to the reports of most wide-spread value for the 
specified topics 


For example: financial management, operational and programme delivery, oversight of 
service delivery, strategic centre of government, accountability in local service delivery and 
performance measurement. 


The ‘Reports by sector’ tab provides access to all reports by sector. 


Departmental Overviews: these guides cover: responsibilities and how developments spend 
their money, financial management, reported performance, issues and areas for performance 
improvement identified in NAO reports, and likely issues in the coming year. 


Support for Audit Committees. 


Self-assessment resources: our ‘how to’ guides and frameworks. 


Technical guidance: materials on technical accounting, audit and internal control information, 
and good practice resources for boards and audit committees 


Search on terms such as ‘briefings’ to find reports 


Work in progress gives a forward look at reports on their way 


Further 
information 


Ways to be kept informed 


NAOdirect 
email alerts 
Follow us 


Ml [infalaya 


Receive NAO’s e:newsletter — 
contact NAO Communications 
if you don’t already receive 
this regularly or would like 

to add someone else to the 
distribution list. 


Ask your NAO team to provide 
you with an “NAO Update” at 
Audit Committee meetings. 


